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A simple permutation is one that does not map a nontrivial inter- 
val onto an interval. It was recently proved by Albert and Atkin- 
son that a permutation class with only finitely simple permuta- 
tions has an algebraic generating function. We extend this result 
to enumerate permutations in such a class satisfying additional 
properties, e.g., the even permutations, the involutions, the per- 
mutations avoiding generalised permutations, and so on. 



1. Introduction 

Substitution decompositions (known also as modular decompositions, disjunctive decom- 
positions, and X-joins) have proved to be a useful technique in a wide range of settings, 
ranging from game theory to combinatorial optimization, see Mohring [30] or Mohring 
and Radermacher [31] for extensive references. Although substitution decompositions are 
most often applied to algorithmic problems, here we apply them to the enumeration of 
permutation classes, a.k.a. restricted permutations. 

The permutation it is said to contain the permutation a, written a < ir, if it has a subse- 
quence that is order isomorphic to a. For example, 7r = 491867532 contains a = 51342, as 
can be seen by considering the subsequence 91672 (= vr(2), 7r(3), 7r(5), 7r(6), 7r(9)), and such 
a subsequence is called a copy of a in 7r. This pattern-containment relation is a partial order 
on permutations. We refer to downsets of permutations under this order as permutation 
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Figure 1: The plots of three simple permutations of length 12. 



classes. In other words, if C is a permutation class, tt £ C, and a < n, then a £ C. Per- 
mutation classes arise naturally in a variety of disparate fields, ranging from the analysis 
of sorting machines (dating back to Knuth [20], who proved that a permutation is stack- 
sortable if and only if it lies in the class Av(231)) to the study of Schubert varieties (see, e.g., 
Lakshmibai and Sandhya [21]). The most frequently investigated property of permutation 
classes is their enumeration. 

We shall denote by C n the set C n S n , i.e. the permutations in C of length n, and we 
refer to ^ as the generating function for C. Recall that an antichain is a set of pairwise 

incomparable elements. For any permutation class C, there is a unique (possibly infinite) 
antichain B such that C = Av(B) = {tt : (3 j£ tt for all (3 E B}. This antichain B, which 
consists of the minimal permutations not in C, is called the basis of C. 

An interval in the permutation tt is a set of contiguous indices I = [a, b] such that the 
set of values tt(I) = {vr(i) : i G /} also forms an interval of natural numbers. Every per- 
mutation tt of [n] = {1, 2, . . . , n} has intervals of length 0, 1, and n; tt is said to be simple 
if it has no other intervals (such intervals are called proper). Figure 1 shows three simple 
permutations. Albert and Atkinson [1] were the first to establish the link between simple 
permutations and the enumeration of permutation classes; they proved that every per- 
mutation class with only finitely many simple permutations has an algebraic generating 
function. Our main theorem implies the following result. (The terms in Theorem 1.1 are 
defined at the end of this section.) 

Theorem 1.1. In a permutation class C with only finitely many simple permutations, the following 
sequences have algebraic generating functions: 

• the number of alternating permutations in C n , 

• the number of even permutations in C n , 

• the number ofDumont permutations of the first kind in C n , 

• the number of permutations in C n avoiding any finite set of blocked or barred permutations, 
and 

• the number of involutions in C n . 

Moreover, these conditions can be combined in any finite manner desired. 
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One class to which this theorem applies is Av(132). In any permutation from Av(132), 
all entries to the left of the maximum must be greater than all entries to the right. This 
shows that Av(132) has only three simple permutations (1, 12, and 21). Thus Theorem 1.1 
helps to explain why there are known formulas and recurrences for enumerating: 

• Av(132, 0) — Mansour and Vainshtein [27], 

• involutions in Av(132, ft) — Guibert and Mansour [16], 

• even involutions in Av(132, j3) — Guibert and Mansour [17], 

• involutions in Av(231, j3) — Egge and Mansour [12], 

• even permutations in Av(132, j3) — Mansour [26], 

• permutations in Av(132) avoiding a blocked permutation — Mansour [22], 

• alternating permutations in Av(132) avoiding a blocked permutation — Mansour [23], 

• Dumont permutations of the first kind in Av(132, 0) — Mansour [25], 

• permutations avoiding 132, 1-23, and the permutation or blocked permutation — 
Elizalde and Mansour [14], 

• permutations in Av(132, /?) that are West-2-stack-sortable — Egge and Mansour [11], 
and 

Moreover, in Brignall, Huczynska, and Vatter [7] a decomposition theorem for simple per- 
mutations is proved which implies that for any fixed r, the class consisting of all permuta- 
tions with at most r copies of 132 contains only finitely many simple permutations. There- 
fore, Theorem 1.1 also implies that the following sets have algebraic generating functions: 

• permutations with at most r copies of 132 — Bona [5] and Mansour and Vainshtein [28], 

• even permutations with at most r copies of 132 — Mansour [24], 

• involutions with at most r copies of 231 — Mansour, Yan, and Yang [29], 

• alternating permutations with at most r copies of 132 — Mansour [23], and 

We conclude this section by defining the terms involved in Theorem 1.1. 

The permutation ir G S n is alternating if for all i G [2, n — 1], ir(i) does not lie between 
Ti{i — 1) and ir(i + 1). A permutation is Dumont of the first kind if each even entry is imme- 
diately followed by a smaller entry and each odd entry is either immediately followed by 
a larger entry or occurs last (this dates back to Dumont [10]). 

A barred permutation is a permutation in which one or more of the entries is barred; 
for 7r to avoid the barred permutation a means that every set of entries of ir that is order 
isomorphic to the nonbarred entries of a can be extended to a set order isomorphic to a 
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Figure 2: The plot of 479832156, an inflation of 2413. 



itself. For example, 51342 avoids 312 because every noninversion (i.e., copy of 12) can be 
extended to a copy of 312 (e.g., prefix the 5), but 51342 contains 213 because the 1 and 
3 are order isomorphic to 23, but there is no way to extend this to a copy of 213. Barred 
permutations have arisen several times in the permutation pattern literature. For example, 
under West's notion of 2-stack sorting (see West [39]) the permutations that can be sorted 
are those that avoid 2341 and 35241, while Bousquet-Melou and Butler [6] characterise 
the permutations corresponding to locally factorial Schubert varieties in terms of barred 
permutations. 

A blocked permutation is a permutation containing dashes indicating the entries that 
need not occur consecutively (in the normal pattern-containment order, no entries need 
occur consecutively). For example, 51342 contains two copies of 3-12: 513 and 534, but 
note that 514 is not a copy of 3-12 because the 1 and 4 are not adjacent. Babson and Ste- 
ingrimsson [4] introduced blocked permutations and showed that they could be used to 
express most Mahonian statistics. For example, the major index of ir is equal to the total 
number of copies of 1-32, 2-31, 3-21, and 21 in ir. 

2. Inflations and Wreath Closures 

Here we briefly review some required terminology and results. Given a 6 S m and nonempty 
permutations ai , . . . , a m , the inflation of a by a\ , . . . , a m — denoted a[a\ , . . . , a m ] — is the 
permutation obtained by replacing each entry a(i) by an interval that is order isomorphic 
to aj. For example, 2413[1, 132, 321, 12] = 479832156 (see Figure 2). Simple permutations 
cannot be deflated. Conversely: 

Proposition 2.1. Every permutation is the inflation of a unique simple permutation. 

For simple permutations of length at least four, one can say more: 

Proposition 2.2. If ir can be written as a[a\, . . . , a m ] where a is simple and m > 4, then the a/s 
are unique. 

In the case where ir = 12[a\ , a^], some caution is needed. A sum indecomposable permu- 
tation is one that cannot be written as 12 [ai, a^], whilst a skew indecomposable permutation 
is one than cannot be written as 2 1 [a\ , 02] ■ 

Proposition 2.3. If tt is an inflation of '12, then there is a unique sum indecomposable a± such that 
ir = 12[«i, 02] for some a 2 - The same holds with 12 replaced by 21 and "sum" replaced by "skew". 
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We refer to the unique decompositions guaranteed by Propositions 2.1-2.3 as the sub- 
stitution decomposition. 

Let Si(X) denote the simple permutations in the set X of permutations. The wreath 
closure, W(X), of X is the largest permutation class with Si(W(X)) = Si(X). For exam- 
ple, the wreath closure of Av(132) is the largest class whose only simple permutations are 
1, 12, and 21. This class is known as the separable permutations 1 , Av(2413, 3142). When 
W(C) = C, C is said to be wreath-closed 2 The crucial property of a wreath-closed class C is 
that a[ai, . . . , a m ] G C for all a, a±, . . . , a m G C. 

3. A More General Theorem 

A property, P, is any set of permutations. We say that tt satisfies P if tt G P. Given a 
permutation class C and set V of properties, we write Cp for the set of permutations in C 
that satisfy every property in V , and write f-p for the generating function of Cp. 

A set V of properties is said to be query-complete if, for all permutations a G S m and 
P G V, it can be decided whether a[a\ , . . . , a m ] satisfies P given only the knowledge about 
whether on satisfies Q, for all i G [m] and Q G V. In other words, this condition means 
that every query "does a[a±, . . . , a m ] satisfy P?" can be answered by considering only 
questions of the form "does «j satisfy Q?" Note that we need only check the condition for 
nontrivial simple a's, as a[a\, ... , a m ] can always be expressed as the inflation of a simple 
permutation (by Proposition 2.1). 

'The separable permutations seem to have made their first appearance as the permutations that can be 
sorted by pop-stacks in series, see Avis and Newborn [3]. Shapiro and Stephens [35] showed that the separable 
permutations are those that fill up under bootstrap percolation. The separable permutations are essentially 
the permutation analogue of series-parallel posets (see Stanley [37, Section 3.2]) and complement reducible 
graphs (see Corneil, Lerchs, and Burlingham [9]). Their enumeration is given by the large Schroder numbers 
(see Footnote 3 or Example 6.1). 

2 It is quite easy to decide if a permutation class given by a finite basis is wreath-closed: 

Proposition 2.4 (Atkinson and Stitt [2]). A permutation class is wreath-closed if and only if each of its basis elements 
is simple. 

One may also wish to compute the basis of W(C). Murphy includes in his thesis [32] an example of a finitely 
based class whose wreath closure is infinitely based. The natural question is then: 

Question 2.5. Given a finite basis B, is it decidable whether W(Av(_B)) is finitely based? 

The analogous question for graphs was raised by Giakoumakis [15] and has received some attention, see 
for example Zverovich [40]. In one notable case Question 2.5 has a nice answer: 

Proposition 2.6. If the longest simple permutations in C have length k then the basis elements ofW(C) have length at 
most k + 2. 

Proof. The basis of W(C) is easily seen to consist of the minimal (under the pattern-containment order) simple 
permutations not contained in C. Let n be such a permutation of length n. Theorem 7.2 shows that n contains 
a simple permutation a of length n — 1 or n — 2. If n > k + 3, then a £ C, so a ^ W(C) and thus tt does not lie 
in the basis of W(C). □ 
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For example, {Av(132)} is not query-complete, as witnessed by the fact that 12[1, 1] G 
Av(132) but 12[1, 21] ^ Av(132). However, {Av(132), Av(21)} is query-complete: 

12[ai,a 2 ] G Av(132) 
21[ai,a 2 ] G Av(132) 
cj[ai,...,a m ] Av(132) 
12[ai,a 2 ] G Av(21) 
<r[ai,...,a m ] ^ Av(21) 

and since 12 and 21 are the only nontrivial simple permutations in Av(132), these are the 
only computations that need to be performed. 
Our main result in this section is the following. 

Theorem 3.1. Let C be a wreath-closed permutation class containing only finitely many simple 
permutations, V a finite query-complete set of properties , and Qcp. The generating function for 
the set of permutations in C satisfying every property in Q, i.e., /q, is algebraic. 

Let us postpone the proof of Theorem 3.1 momentarily in order to introduce notation 
and review pertinent facts. We begin by considering the case V = 0, which contains many 
of the main ideas of the proof in a more digestible form. We introduce two properties, 

= {sum indecomposable permutations} and 
= {skew indecomposable permutations}. 

Note that both {0} and {0} are query-complete, because for simple a, 

a[ai,...,a m ] G a ^ 12. 

We also introduce the notation 

cr[Ci, . . . ,C m ] = {a[ax, . . .,a m ] : a; G C, for all i G [m]}. 

By Propositions 2.1-2.3 and the assumption that C is wreath-closed, C can be written as 

C = {1}«12[C0,C]«21[C ,C]W [+j a[C,...,C], 

crGSi(C) 
|cr|>4 

while Ccg and C0 have the expressions 

C0 = {1}W21[C0,C]W (+J a[C,...,C] = C\12[C0,C], 

o-eSi(C) 
M>4 

C0 = {1}W12[C0,C]W 1+J a[C,...,C] = C\21[C0,C). 

o-esi(e) 

\a\>4 



=^ cti G Av(132) and a 2 G Av(21), 
=^ ai G Av(132) and a 2 G Av(132), 
if o- i {1,12,21}, 

ai G Av(21) and a 2 G Av(21), 
if a ^ {1,12}, 
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These give the system 



/ 



x + f0f + f0f+ E 



r 



o-eSi(C) 
kl>4 



/0 



^ + /0/+ E / 



/ 



< 



o-esi(c) 

|cr|>4 



1 + / 




x + / /+ e / 



/ 



o-eSi(C) 

kl>4 



1 + /' 



If we now let s denote the generating function for the simple permutations of length at 
least 4 in C, we get that 



In general the situation will not be so straightforward and we will resort to algebraic 
systems. The following is a specialisation of the more general treatment in Stanley [38, 
Section 6.6]. 

Let A = {ai, . . . , a n } denote an alphabet. A proper algebraic system over Q[x±, . . . , x m ] is 
a set of equations a, = Pi{x\, . . . , x m , a±, . . . , a n ) where each pi is a polynomial with coeffi- 
cients from Q, has constant term 0, and contains no terms of the form ca-i where c G Q. The 
solution to such a system is a tuple (/i, . . . , /„) of formal power series from Q[[xi, . . . , x m )] 
such that for all i, U is equal topi(xi, . . . , x m , ai, . . . , a„) evaluated at (ai, . . . , a n ) = (/i, . . . , /„). 

Theorem 3.2 (Stanley [38, Proposition 6.6.3 and Theorem 6.6.10]). Every proper algebraic 
system (pi, . . . ,p n ) over Q[x\, . . . , x m ] has a unique solution (/i, . . . , /„). Moreover, each of these 
fi's is algebraic over Q[x±, . . . , x m \. 

Proof of Theorem 3.1. If "P is query-complete then V U {0, 0} is also query-complete, so we 
may assume without loss that 0, G V. 

Let V(ir) denote the set of properties in V satisfied by ir and let gn denote the generating 
function for the set of tt G C with V(tt) =1Z, so 



As V is query-complete, for each simple a G C of length m there is a finite collection of m- 
tuplesof sets of properties such that V(a[a\, . . . ,a m ]) = 1Z precisely if (V(ai), . . . ,V(a m )) 

3 In particular, note that the separable permutations correspond to s = 0; making this substitution leaves 
/ = x + 2/ 2 /(l + /), giving the large Schroder numbers. 
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lies in this collection. If m > 4 then Propositions 2.1 and 2.2 imply that the generating func- 
tion for all inflations ir of a with V(ir) = 1Z can be expressed nontrivially as a polynomial 
in {gs : S C V}. If m = 2, suppose a = 12 without loss. By Proposition 2.3, all inflations of 
12 have a unique decomposition as 12[ai, a?\ where a\ £ 0. Thus the generating function 
for inflations 7r of 12 with V(ir) = 1Z can be expressed as a sum of terms of the form gsQT 
where G 5. 

Therefore (77?. can be expressed as a polynomial in x (depending on whether V(l) =TV) 
and {gs : S C T 3 }. Moreover, these polynomials have no constant terms and no terms of 
the form cg$ for constant c / 0. Thus they form a proper algebraic system, so Theorem 3.2 
implies that each gs is algebraic. □ 

We are now in a position to indicate the proof of Theorem 1.1, which will be completed 
over the course of the next two sections. Suppose that C contains only finitely many simple 
permutations and let B denote the basis of C. A result of Higman [19] implies that C is 
finitely based, i.e., B is finite; see Albert and Atkinson [1] for details. By the forthcoming 
Lemma 4.1, each property Av(/3) lies in a finite query-complete set. Clearly the union over 
all (3 € B of these finite query-complete sets is again a finite query-complete set, and we 
have 

C = W(C){Av(/3) : /3eB} ) 

so the generating function for C is algebraic by Theorem 3.1. Now suppose that we are 
interested in the even permutations in C. In this case we let EV denote the set of even 
permutations. Lemma 4.3 shows that EV lies in a finite query-complete set, so the even 
permutations in C, 

C{EV} = W(,Q{EV}U{Av(f3):f3£B}, 

also have an algebraic generating function by Theorem 3.1, thereby establishing that part 
of Theorem 1.1. The other cases — with the exception of involutions, discussed in Section 5 
— are analogous. 

4. Finite Query-Complete Sets 

In order to utilise the general framework provided by Theorem 3.1, we need to construct 
finite query-complete sets of properties. We do that here, beginning with the most impor- 
tant one. 

Lemma 4.1. For every permutation 0, the set {Av(<5) : 5 < /3} is query-complete. 

Proof. By applying the lemma to {Av(7) : 7 < 5}, we see that it suffices to prove that 
whether tt = a[a±, . . . , a m ] £ Av(/?) can be decided entirely by knowing, for each i, which 
permutations S satisfy S < a, L and S < (3. 
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We define a lenient inflation to be an inflation a[a±, . . . , a m ] in which the a/s are allowed 
to be empty. List all expressions of (3 as a lenient inflation of a as 

/? = a[$\...,(3$], 
(5 = *\$\...,p®]. 

Clearly if we have, for some s G [t], ai > (3^ for all i G [m], then it > f3. Equivalently, to 

have 7r G Av(/3), for every s G [t] there must be at least one i G [m] for which ai ^ /3j- s \ 
Conversely every embedding of (3 into 7r gives one of the lenient inflations in the list above, 
completing the proof. □ 

The proof of Lemma 4.1 extends in the obvious manner to show that the property of 
avoiding a blocked or barred permutation (or, for that matter, a permutation combining 
these restrictions) also lies in a finite query-complete set. 

Lemma 4.2. The set of properties consisting of 

• AL = {alternating permutations}, 

• BR = {permutations beginning with a rise, i.e., permutations with ir(l) < tt(2)}, 

• ER = {permutations ending with a rise}, and 

• {!}• 

is query-complete. 

Proof. Clearly {{1},BR, ER} is query-complete: 

a[a\, . . . , a m ] G BR <?=^ a\ G BR or («i = 1 and a(l) < a(2)) , 
a[ai, . . . , a m ] G ER <^=^ a m G ER or (a m = 1 and a(m — 1) < a(m)) . 

For a[a\, . . . , a m ] to be an alternating permutation, we first need a>i, . . . , a m G AL. 
Now note that if a(i) < a(i + 1) then a[ai, . . . , a m ] will contain a rise between the interval 
corresponding to a{i) and the interval corresponding to a(i + 1). Thus «j must end — and 
aj + i must begin — with a descent (or have length 1), so we need ^ ER and aj+i ^ 
Si?. Similarly, if a(i) > a(i + 1) then we need G ER U {1} and a m e BRU {1}. 
These are easily seen to also be sufficient conditions for a[a>x, . . . , a m ] to be an alternating 
permutation, completing the proof. □ 

Lemma 4.3. The set of properties consisting of 

• EV = {even permutations} and 

• EL = {permutations of even length} 
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is query-complete. 



Proof. We have 



a[ai, . . . , a m ] G EL 



an even number of a/s fail to lie in EL, 



so {EL} is query-complete. To see that {EV, EL} is query-complete, we divide the inver- 
sions in cr\a\, . . . , a m ] into two groups: inversions within a single block aj and inversions 
between two blocks ai and aj. We need to compute the parity of each of these numbers. 
The parity of the number of inversions within a% is given by whether ai G EV. The parity 
of the number of inversions between the blocks on and aj, i < j, is even if a{i) < cr(j), if 
oti G EL, or if aj G EL, and odd otherwise. □ 

Lemma 4.4. The set of properties consisting of 

• DU = {Dumont permutations of the first kind} and 

• EL = {permutations of even length} 
is query-complete. 

Proof. It suffices to determine which entries of a[a\, . . . , a m ] have even value and which 
have odd value, and this can be decided based on the knowledge of which q/s have even 
length. □ 

5. Involutions 

We have now proved all of Theorem 1.1 except for the claim that we can count the invo- 
lutions in C (and even involutions, alternating involutions, etc.). Unfortunately, involu- 
tionhood lies just outside the scope of our query-complete-property machinery: letting / 
denote the set of involutions we have that 12[ai, 02] G / <^=^ a\, ai G /, but when is 

21[ai,a 2 ] G /? 

We begin by considering the effect of inversion on the substitution decomposition. First 
note that if tt is an inflation of a then 7r _1 is an inflation of a^ 1 . Recalling Proposition 2.1 
("every permutation is the inflation of a unique simple permutation"), we have that if 7r is 
an involution then it must be the inflation of a simple involution. By Proposition 2.2 we 
then get the following: 

Proposition 5.1. If tt = a[ai, . . . , a m ] is an involution and a is a simple permutation of length 
at least 4, then a is an involution and ai = a~^for all i G [m]. 

As in Section 2, a = 12 and a = 21 must be handled separately. Moreover, when 
7r = 21 [ai, 0:2], unless a\ and a^ are both skew indecomposable, we use the middle greedy 
decomposition, expressing tt as 321[ai, 02, 03] with «2 as long as possible. 
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Proposition 5.2. The involutions that are inflations of permutations of length 2 are precisely those 
of the form 

• 12[a\, 02] for involutions a± and ct2, 

• 21 [ai, 02] for skew indecomposable a± and «2 with a.\ = a^ 1 , and 

• 321 [«i, «2, 03]/ where ct\ and 03 are skew indecomposable, a\ = a$ , and 02 is an involu- 
tion (i.e., the middle greedy decompositions). 

Define the inverse of the property P by P^ 1 = {vr^ 1 : it G P}, and for a set of properties 
V,V~ l = {P- 1 : P G V}. 

Theorem 5.3. Let C be a wreath-closed permutation class containing only finitely many simple 
permutations, V a finite query-complete set of properties, and Qcp. The generating function for 
the set of involutions in C satisfying every property in Q is algebraic. 

Proof. We assume (without loss) both that 0, G V and that V = V~ x . As in the proof 
of Theorem 3.1, let V(tt) denote the set of properties in V satisfied by ir and g-ji denote the 
generating function for the set of tt G C with V(ir) = 1Z. Also let h-ji denote the generating 
function for the set of involutions ir G C with V(ir) = 1Z. It suffices to show that each hn is 
algebraic over Q[x]. 

As Propositions 5.1 and 5.2 indicate, we need to count pairs (a, a -1 ) where a and a~ 1 
satisfy certain sets of properties. To this end define 

pk= E * H+|or11 - 

V(a)=U 

(Note that if V(a) = PL then V(oT v ) = Pr l because V = V' 1 .) 

Now take a to be a simple permutation. We need to compute the contribution to hn 
of inflations of a. As before, since V is query-complete, V(a[a\, . . . , a m \) = 1Z if and only 
if (V(ai), . . . ,V(a m )) lies in a certain collection of m-tuples of sets of properties. Choose 
one of these m-tuples, say (TZ\, . . . , lZ m ), and suppose first that m = \a\ > 4. It suffices to 
calculate the contribution of involutions of the form a[a±, . . . , a m ] with Vipn) = 1Zi for all 
i G [m]. If there is some j G [m] for which TZj / TZ^u\ then this contribution is 0. Otherwise 
the contribution is a single term in which each fixed point j corresponds to an factor 
and each non-fixed-point pair (j, corresponds to a pn factor. A similar analysis of 
inflations of 12 and 21 — in the latter case using the middle greedy decomposition when 
necessary — allows us to compute the contribution of these inflations. 

Therefore each h-ji can be expressed nontrivially as a polynomial in x, {hs : S C V}, 
and {ps ■ S C V}. Viewing x and {ps ■ S C V} as variables, Theorem 3.2 implies that 
each hn is algebraic over Q[x, {ps : S C V}]. However, ps is nothing other than gs(x 2 ), so 
Q[ x , {ps '■ <5 C V}] is an algebraic extension of Q[x] by Theorem 3.1, proving the theorem. 

□ 
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One could adapt Theorem 5.3 to count the permutations in C that are invariant under 
other symmetries. For example, permutations invariant under the composition of reverse 
and complement have received some attention, see Guibert and Pergola [18]. 

6. Examples 

While we have already shown how to enumerate separable permutations in Footnote 3, 
here we use the approach of Theorem 3.1. 

Example 6.1: Separable permutations. With the notation from the proof of Theorem 3.1, 
we have that for the separable permutations: 

90,0 = 

90 = (90,0+90)(90,0+90+90), 
90 = (90,0 +90^90,0+90+ 90)- 

We are interested in / = g0 + g0 + g0. We can find the algebraic equation satisfied by 
/ by computing the Grobner basis (under an appropriate monomial ordering) of the ideal 
generated by 

90,0 -x, 

90 ~ (90,0 + 90)(90,0 + 90 + 90), 
90 ~ (90,0 + 90)(90,0 +90+ 90), 
f -90,0 +90+90- 

This calculation shows that 

f 2 + (x-l)f + x = 0, 

so 

1 — x — Vl — 6x + x 2 
* ~ 2 ' 

This, reassuringly, is the generating function for the large Schroder numbers. 

This system does not change dramatically when another simple permutation is intro- 
duced, as shown by the next example. 

Example 6.2: The wreath closure of 1, 12, 21, and 2413 4 . Here the system is 

90,0 = x + (90,0+90+90) A , 
90 = (90,0 + 90)(90,0+90+90), 
90 = (90,0 + 90)(90,0 +90+ 90)- 

Setting 

/ = 50,0 +90+90 

"Using Proposition 2.6 it can be computed that this is the class Av(3142, 25314, 246135, 362514). 



Simple Permutations and Algebraic Generating Functions 



13 



and computing the Grobner basis for the associated ideal as in the previous example shows 
that the generating function for this class satisfies 

f + f + f 2 + (x-l)f + x = 0. 

Example 6.3: Av(132). The wreath closure of Av(132) is the class of separable permuta- 
tions, so to enumerate Av(132) we need to refine Example 6.1. While Proposition 4.1 shows 
that {Av(l), Av(12), Av(21), Av(132)} is query-complete, we remarked at the beginning of 
Section 3 that {Av(21), Av(132)} will suffice. Our system is then 

9$,0,Av(21) = x > 

15'0,Av(21) = 5'0,0,Av(21)(fi'0,0,Av(21) + fl , 0,Av(21))> 

50 = (50,0,Av(2i) + fl r 0,Av(2l) + 5'0)(5 f 0,0,Av(2i) + 90,Av{2l) + 50 + 90), 

90 = 90(90,0,Av(21) +90,Av{21))- 

(As we are only interested in 132-avoiding permutations we have suppressed the subscript 
Av(132), which would otherwise be present in all these terms.) Setting 

/ = 90,0,Av{2i) + 90,Av(2i) +90+90 
and computing a Grobner basis as before yields 

_ 1 - 2a - VI - 4x 
1 ~ 2~x ' 

the generating function for the Catalan numbers, as expected. 
Example 6.4: Av(2413, 3142, 2143). Our system in this case is 

90,0,Av(2l) = x , 

fi , 0,Av(21) = fi , 0,0,Av(21)(5 , 0,0 l Av(21) + 50 l Av(21))' 

90 = (50,0,Av(2i) + 50,Av(2l) + 90)(90,0,Av(2i) + 90,Av(2l) +90+ 90), 

k 90 = #0,0,Av(2i)(50 +90) +50(50,0,Av(2i) +50,Av(2i))> 

where here we have suppressed the Av(2143) subscript. Solving as before shows that the 
generating function for this class is 

1 - 3x + 2x 2 - Vl - 6x + bx 2 
2x{2 - x) ' 

and thus the number of permutations of length n in this class is ^ (fyfn-k where (/„) 
denotes Fine's sequence. 

Example 6.5: Alternating separable permutations. Lemma 4.2 shows that we need to 
introduce the properties AL (alternating permutations), BR (permutations beginning with 
a rise), ER (permutations ending with a rise), and {1}. In the separable case {1} = n 
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so we can ignore that property, and as AL occurs in each of the terms of our system we 
suppress it. We then have 

' 90,0 = x, 

90 =(#0,0 + 90,er) (90,0+90,br+90,br)> 

90,br = 90,br,er(90,0 + 90,BR + 90,br)i 

90, er = (90,0 + 90,er)(90,br,er + 90,br,er)> 

< 90,BR,ER = 90,BR,ER(90,BR,ER~ >r 90,BR,Er)i 

90 = 90(90 + 90), 

90,BR = (90,0 + 90,br)(90+90), 

90,er = 90(90,0 +90,er + 90,er)> 

> 90,BR,ER = (90,0 + 90,br)(90,0 + 90,er + 90,er)- 

The generating function for these permutations satisfies 

f - (2x 2 -5x + 4)/ 2 - (Ax 3 + x 2 - 8x)f - (2x 4 + 5x 3 + Ax 2 ) = 0. 

Example 6.6: Separable involutions. Using the notation from the proof of Theorem 5.3, 
we wish to find / = hg£ s + hM + h0. These generating functions are related to each other 
and to the p generating functions by 

^0,0 = x > 

h = (P0,0+P0) + (P0,0+P0)( h 0,0 + h + h 0)> 
h = ( h 0,0 + h 0)( h 0,0 + h + h 0)- 

From Example 6.1 it can be computed that 

P0,0~ x2 = °> 
2p 2 + (3x 2 - 1) P + x 4 = 0, 

2p 2 + {3x 2 - 1) P + x 4 = 0. 
Combining these with the system above and solving as usual shows that 

x 2 f + O 3 + 3x 2 + x - l)f + (3x 3 + 6x 2 - x)f 2 + (3x 3 + 7x 2 - x - l)f + x 3 + 3x 2 + x = 0. 



7. Concluding Remarks 

We end by discussing issues related to the applicability and application of these tech- 
niques. 

Determining if these methods apply. The results we have established apply only to per- 
mutation classes with finitely many simple permutations. Thus it would be useful to be 
able to determine whether a permutation class contains finitely many simple permuta- 
tions. This can be done: 
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Theorem 7.1 (Brignall, Ruskuc, and Vatter [8]). It is decidable whether permutation class given 
by a finite basis contains only finitely many simple permutations. 

Finding the simple permutations. Thus far we have tacitly assumed that the set of simple 
permutations in our class is known. Since classes are often specified by their bases, this set 
of simple permutations must first be computed. Assuming that this set is finite, it can be 
computed via a result of Schmerl and Trotter. While we state only the permutation case 
(a proof of this case is also given by Murphy [32]), their result covers all irreflexive binary 
relational structures. See Ehrenfeucht and McConnell [13] for a version of this theorem for 
certain other relational structures. 

Theorem 7.2 (Schmerl and Trotter [34]). Every simple permutation of length n > 2 contains a 
simple permutation of length n — 1 or n — 2. 

For example, the number of simple permutations in Av(1324, 2143, 4231) of lengths 1 to 
7 is 1,2,0,2,4,0,0. Because there are no simple permutations of length 6 or 7 in this class, 
Theorem 7.2 ensures that it contains no longer simple permutations. 

Derangements. Notably absent from Theorem 1.1 is the number of derangements in C, 
despite the fact that 132-avoiding derangements are counted by Fine's sequence (Robert- 
son, Saracino, and Zeilberger [33]). To see that the set of derangements does not lie in a 
finite query-complete set of properties, for a G S n define D(a) = {a(i) — i : i G [n]}. Then 
21 [12 • • • j, a] is a derangement if and only if j ^ D(a). This shows that a± and ot2 must lie 
in different sets of properties whenever D(a\) nff ^ D{oi2) H N, implying that the set of 
derangements can only lie in an infinite query-complete set of properties. 

Other structures. It is natural to expect the substitution decomposition to lead to enumer- 
ative results for other types of combinatorial object. Indeed, Stanley [36] used a method 
similar to the substitution decomposition to count unlabelled series-parallel posets, those 
generated from the singleton poset 1 by ordinal sum and disjoint union. While in the per- 
mutation case we are fortunate to be able to ignore isomorphism concerns, this is not the 
case for series-parallel posets, which do not have an algebraic generating function (Stanley 
gives a functional equation satisfied by their generating function). 

Acknowledgements. We thank Mike Atkinson, Mireille Bousquet-Melou, and Steve Lin- 
ton for their helpful comments and suggestions, and John McDermott for his technical 
expertise. 
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